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(54) Title: DEFECT CHANNEL NULLING 



(57) Abstract 

; A method of and apparatus for removing the effects of surface and near surface image (12) storage media (14) defects from a scanned 

! image using an infrared record as a norming control. Each pixel in a visible channel of the scanned images is divided by the corresponding 

I pixel in the associated infrared control channel after it has been altered in gain to match the degree of defect in the visible channel. By 

I appropriately altering the gain prior to dividing the pixel information, imbalances between the visible and infrared records which would 

j leave defect residue after the division are reduced or eliminated. To remove defect residue, a degree of nulling is established for each defect 

■ region based on the visible and infrared content in that region. In one embodiment, the articulation gain is multiplicatively applied to the j 

i logarithm of the visible and infrared records. 
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DEFECT CHANNEL NULLING 

RELATED APPLICATION 

5 This application claims the benefit of U.S. Provisional Application No. 

60/035,763, filed January 6, 1997. 

FIELD OF THE INVENTION 

This invention relates to electronic image enhancement and recovery, and more 
10 particularly to a method and apparatus which compensate for the effects of defects of the 
media on which the image is stored. 

BACKGROUND OF THE INVENTION 

Ever since the first image of an object was captured on film, a serious problem 
15 became apparent: imperfections in and on the recording medium itself which distort and 
obscure the original image that was sought to be captured. This non-image imperfection 
problem continues to plague the field of image capture and reproduction to the present day. 
These imperfections occur in many forms including dust, scratches, fingerprints, smudges, 
and the like. 

20 The film industry has been concerned that the problem caused by these non-image 

imperfections may jeopardize the long term future of analog images. Notwithstanding the 
significant efforts that have been made to solve the problem, it nevertheless persists. This 
is particularly true with respect to enlargements and high resolution scans. Thus, the 
problem is becoming even more acute and recognized as resolutions increase. 

25 Furthermore, multimedia applications have brought widespread attention to the problem 
with the increase in film scanning for computer applications. 

The non-image imperfection problems occur more frequently with negatives than 
with transparencies because (1) viewing the negative requires a light to dark ratio gain (also 
known as a "gamma") of .about two to one; whereas viewing a transparency is less 

30 demanding and (2) filmstrips are subject to more contact than are mounted transparencies. 
Some imperfections may be present in fresh photographic film, for example, media surface 
waves or microbubbles in the emulsion. Improper processing may also introduce 
non-image imperfection problems, for example mild reticulation or residual unbleached 
silver. Some non-image imperfection problems are introduced through improper handling 

35 of the image media which may leave oily fingerprints and other surface contaminants or 
cause physical damage such as scratches which distort the view of the image. Improper 
storage may also leave the film further exposed to defect inducing environmental influences 
such as dust and chemicals. Archival polypropylene sleeves employed to protect negatives 
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contribute to the problem by leaving hairline surface scratches as the negatives are pulled 
out and replaced into the sleeves. As illustrated by the above examples, non-image 
imperfections may arise out of one or more of the following: film emulsion defects; 
improper processing; and improper handling or storage which may cause extraneous matter 
5 accumulation or scratches on the image media. 

Numerous measures have been developed in the art in an attempt to address these 
problems, particularly with respect to captured high resolution images. Generally 
speaking, these measures are in the nature of prevention and labor intensive minimization of 
non-image defects. One approach has been that of preventing introduction of non-image 

10 defects in the development process itself. Expensive anti-static equipment, including spray 
machines to neutralize dust attracting charges, are employed by some photo finishers. 
Photo finishers also attempted to solve some of the problems by employing diffuse light 
source enlargers that help reduce the effects of refraction of the light due to imperfections in 
the photo finishing processes. 

15 Another approach used in the an involves minimizing the effects of these 

imperfections once they are present by various correction techniques, most of which are 
manual and thus highly labor-intensive and expensive. For example, during the photo 
finishing process, a highly trained individual might spend a great deal of time using various 
spotting dyes and an extremely small spotting brush in an effort to essentially paint out the 

20 imperfections in a reproduced or printed image. Another technique is to wipe on or 
immerse the negatives in a light oil in an attempt to optically fill scratches. 

As the art has developed, various attempts have been made to automate the 
correction process, particularly with respect to digital image systems. In such systems, 
once an imperfection has been detected, various "fill" algorithms are used to correct the 

25 image at the imperfection site. Nevertheless, heuristics or human intervention has been 
required to detect the imperfections with a subjective threshold. Typically the area 
identified to be corrected in this manner is much larger than necessary, in part due to 
application of these subjective criteria for detecting defective areas. 

Automated methods have also been developed for detecting imperfect areas in 

30 recording media, as described in German patent 2821868.0 published Nov. 22, 1979, and 
entitled "Method and Device for Detecting Recording and Counting of Mechanical Damage 
to Moving Bands, for Example Films." The approach discussed focuses on determining 
the quantity of defects and shutting down processing if that measured quantity exceeds 
some predetermined maximum level of defects. In this system a source of infrared energy 

35 impinges upon the film medium. A scanned infrared image of the film in question is then 
taken by sensors detecting reflection of the infrared energy from the film surface. 
However, several limitations are present in the system disclosed by the patent 
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First, its purpose is not to correct the effects of such detected film defects present on 
the film image. Instead, the system is implemented simply to monitor the prevalence of 
these defects in an automated photographic development process whereby the process can 
be automatically shut down if the defect rate exceeds a prescribed level. The optical 
5 infrared path is a reflective one from the infrared source to the infrared sensor which is 
typically different from the other sensors utilized for image processing. The infrared image 
is not recorded in registry with any other scanned images from the visual portion of the 
electromagnetic spectrum. Registry refers to the precise alignment of two related images, 
plates, or impressions so that they are held in position relative to each other. The fact that 

10 this prior an system does not record the infrared image in registry with any other images 
taken of the film is a disadvantage which in and of itself renders it extremely difficult to 
subtract out the effect of such imperfections noted in the infrared image from similar defects 
present in the visual record of the image. 

In another prior system disclosed by the present inventor, a method which 

15 compensates for the effects of storage media defects on image data is disclosed by deriving 
from the medium separate images in the red, green, blue and infrared portions of the 
electromagnetic spectrum, each corresponding to the image stored therein. As disclosed in 
U.S. Patent No. 5,266,805 issued to the present inventor on Nov. 30, 1993, red, green, 
blue and infrared light is sequentially directed at one side of the film by means of a light 

20 source and color filter wheel. Corresponding red, green, blue, and infrared images formed 
by that portion of the light being transmitted through the film are digitally captured from the 
opposite side of the film. The images are preferably captured in registry to facilitate 
subtracting out the effect of imperfections at locations in the infrared record from 
corresponding locations in the red, green, and blue images. The imperfections may either 

25 substantially reduce or totally occlude the infrared light. However, remaining portions of 
the medium having the desired image without such imperfections are essentially uniformly 
transmissive to infrared light. These imperfection-free portions have variable transmissivity 
in the visual spectrum which is determined by the image developed on the film. 
Accordingly, the infrared image may serve as an indicator or map of the spatial position of 

30 these non-image imperfections on and in the medium, thereby allowing determination of the 
location and removal of the defects so that the underlying desired image may be recovered. 

In order to remove film defects from a scanned film image, an infrared image record 
is used as a norming channel by dividing each pixel in a visible channel (red, blue, or 
green) by the corresponding pixel in the associated infrared channel. Although this 

35 improves the image, variations in scanner contrast across the density range of the film, 
variations in image focus, variations in illumination aperture and variations in the way 
defects respond to infrared light cause imbalances between the visible and infrared records 
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array are increased, the width and height of the image are calculated as a function of the 
levels in the array at step 84. This process is repeated as long as the width and height of the 
current level are not both equal to one as checked by condition 86. When the width and 
height of the image are both equivalent to one at decision step 88 thereby indicating a 
5 pyramid level consisting of a single pixel, the counting of the number of levels is completed 
and is set by step 90. With this step, variable initialization is complete. 

In one embodiment, the next step is to convert the image and defect records into 
logarithmic format. Converting to a log format is desirable because of the pyramid building 
process which facilitates splitting the images into frequency bands. Each layer of the 

10 pyramid represents a frequency band. Using the pyramid process allows a zero expected 
value, or mean, at each level and therefore eliminates the bias or "DC* term described 
earlier that differentiates a covariance and correlation. Because a zero mean can be reached, 
a linear format which could possibly result in a division of zero by zero (if the visible 
record is differentiated from the defect record by a division) cannot be used. Converting to 

15 a logarithmic scale solves this problem because division of the image record by the defect 
record is effected by a simple subtraction between equivalent levels of the image and defect 
pyramids. Thus, use of the log domain combined with the pyramid process solves 
problems both with DC bias in the correlation calculations and with effective division on 
separate frequency bands. 

20 When using the disclosed method to null out defects, it is helpful to be able to 

measure how the spatial frequency response of the defects in one record compares to the 
response in another record. One way to do this is to perform a fast Fourier transform on 
each record and compare them. Based on the comparison, the attenuated frequencies of one 
record can be boosted to match the other. Another expedient is to perform two-dimensional 

25 cross correlation and autocorrelation calculations. By spatially dividing one by the other, 
relative blurring in the image may be found and corrected. In a preferred embodiment, each 
image is divided into frequency bands. By performing the nulling process on each 
frequency band separately, any variations in frequency response between the images will be 
automatically compensated thereby boosting any attenuated frequencies. 

30 One way of dividing the frequency into frequency bands that allows the original 

image to be reassembled is to create a band pass pyramid. In a bandpass pyramid, the 
image is first low pass filtered and then subsampled at a lower resolution. The lower 
resolution is then upsampled to the original resolution in order to produce an image devoid 
of details that could not be represented by the lower sampling frequencies but which 

35 contains all the details that could be represented by the lower frequencies. When this 
upsampled image is subtracted from the original image, only the high frequency details 
remain. The entire process is repeated on the subsampled image just discussed and the 
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from the corresponding image level, the defect level is multiplied by a gain chosen to match 
the defect part of the image to provide a better null after subtraction. Preferably, this gain is 
made a function of position within the defect record so that each element of the defect 
record can be adjusted with a different gain. 
5 There are many ways of finding this gain. One way is to iteratively try different 

gains until detail in the subtracted image record does not match detail in the defect record. 
In other words, the gain is chosen based on which one makes the image record and defect 
record as uncorrelated as possible. The defect channel nulling of the present invention uses 
a non-iterative method involving direct calculation to reach an adequate degree of detail 

1 0 uncorrelation. One measurement of the detail matching is the mathematical crosscorrelation 
in which the two records are multiplied element by element. To the degree both records 
match, their detail will track each other closely, in either case producing a positive result 
after the element multiplication. To the extent that they are uncorrelated, the 
crosscorrelation multiplication will sometimes produce a positive product, but will just as 

15 likely produce a negative product. Thus, when the crosscorrelation is averaged over a 
region, any nonzero value indicates a match or correlation. 

Other refinements in gain are based on prior knowledge of record characteristics. 
For example, in one embodiment, the image record is set to zero whenever the defect rises 
above a predetermined intensity. Setting the image record to zero when the defect intensity 

20 is high is an acknowledgement that beyond a certain point, an accurate prediction based on 
the image is impossible, and the best estimate is just zero. Although it is possible to set the 
image record pyramid level to zero whenever the magnitude of the defect record exceeds a 
threshold, the present embodiment uses instead a refinement that allows a gradual zeroing 
or blending so that no abrupt drops to zero are created in the image record. Another 

25 possible refinement approach includes varying the size of the averaging area based on 
image and defect characteristics. Thus, the gain refinement used by the present 
embodiment is given as an example, and is not intended to limit the process to a specific 
mode of calculating the gain. 

The next step in the defect nulling process is to correlate the bandpass frequency 

30 band for both the image and the defect pyramids. An overall description of the correlation 
process is illustrated in Figs. 9A and 9B. The correlation process begins with the 
generation of a correlation histogram at step 156 for each corresponding level in the image 
and defect pyramids. Figs. 9A and 9B give a detailed description of how to generate a 
correlation histogram for a single level of the image and defect pyramids. Processing starts 

35 with each image pixel and the corresponding defect pixel at a specified level. First, 
variables used in the process are initialized at step 158. Specifically, the inner limit for the 
histogram is set to zero and the outer limit is set to two. These limits are chosen to center 
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around the ideal situation when the correlation equals one. Because the correlation is a 
squared number, its lowest value is zero, thereby setting the inner limit for the histogram. 
In order for the approach to be balanced around one, the upper limit is set at two. 
However, it is understood that other limits could be used. 
5 After the histogram is generated, the next step 160 is to calculate the correlation 

cutoffs which are the new upper and lower cutoff limits for the histogram. 

A graph of the histogram with the correlation cutoffs is illustrated in Fig. 11. This 
figure shows how well the defect record tracks the visible component of a defect. The 
x-axis 162 is the ratio of the defect record to the visible record when the visible image has a 

10 defect. The y-axis 164 is the number of pixels representing that ratio weighted by the 
square of the pixels. If, for example, the visible defect exactly tracked the defect record, all 
pixels would show a one to one correspondence. In practice, however, the infrared and 
visible records do not track exactly, and the graph of Fig. 1 1 shows the degree to which 
they diverge. The ten percent points 166 and 168 as illustrated attempt to show the range 

15 for which articulation or fine adjustment of defect gain relative to visible is expected to be 
needed. The process of calculating the correlation cutoffs is described in Figs. 12A and 
12B. These new limits 166 and 168 define the range within which the values will be 
considered. 

After the cutoff values are defined, a crosscorrelation and autocorrelation are 

20 calculated as illustrated in Fig. 13. Applying the cutoff values from Fig. 12B to the image 
(also referred to as preclamping) produces a clamped image in which each individual image 
element is limited to a present value. Fig. 14 illustrates the process of preclamping which is 
a way of expressing the importance of a particular pixel. Normally, it is expected that a 
defect in the visible record is about 1.0 times the defect record. The purpose of nulling is to 

25 accommodate those cases that deviate from 1.0. The present embodiment preclamps the 
visible record to between 0.0 and 2.0 times the defect record prior to taking the 
crosscorrelation and autocorrelation. In this way, the effect of large variations in the visible 
record are taken into account but are limited in value before the multiplication of the 
crosscorrelation and autocorrelation are performed. In Fig. 14, this is done by setting the 

30 variables Low Clamp equal to 0.0 and Upper Clamp equal to 2.0. If the visible record 
were, for example, 4.0 times the defect record, the preclamping would treat the situation as 
though it were only 2.0 times the defect record, but would maintain full weighting, which 
is mathematically equivalent to letting it be 4.0 while only counting it with half the weight. 
Another weight which can be used to express confidence in a pixel is based on how far the 

35 calculated correction coefficient (defined below) varies from the expected value of unity for 
each individual pixel. If the variation from unity is far, that pixel is given little weight and 
vice versa. In a specific example, any pixel that gives a single pixel correction coefficient 
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between 0.0 and 2.0 is given full weight. For a correction coefficient beyond these values, 
the preclamping process effectively reduces weighting of the particular pixel proportional to 
just how far the correction coefficient is beyond these limits. Upon completion of the 
process described in Fig. 14, the clamped element rather than the element itself is then used 
5 to calculate the crosscorrelation and autocorrelation. 

As shown in Fig. 13, the crosscorrelation between the image and the defect record 
is, for a single pixel, the product of two values, the clamped element multiplied by the 
defect element. The defect autocorrelation for the same pixel is the defect element value 
squared. If the crosscorrelation is then divided by the autocorrelation for that single pixel, 

10 the result, in theory, is the correction coefficient which is the amount of the defect record 
present in the image record. However, if this approach is strictly applied, the entire image 
will be nulled to zero for each pixel, which is obviously undesirable. Instead, only that 
component of the image that matches the defect should be nulled. This is done by 
averaging these values over a region. This averaging process or "smudging" is described 

15 in Fig. 9B. In one embodiment of the invention, a 3(3 element area is used for the 
averaging region because it encompasses an average full wave in the pyramid frequency. A 
larger area would give a more accurate estimate of the correction coefficient; however, 
smaller areas allow better articulation around the edges of defects which, because of 
imperfections in the angle of lighting between different colors, may require the correction 

20 coefficient to vary pixel by pixel. 

After the single element cross and autocorrelations for each element of the 3(3 
element region are calculated beginning at step 190, the nine cross correlations are averaged 
and the nine autocorrelations are averaged at step 192. As previously mentioned, this 
averaging process across a region is called smudging. The average of the cross correlations 

25 is termed the regional crosscorrelation value. The average of the autocorrelations is the 
regional autocorrelation value. Finally, the regional crosscorrelation divided by the regional 
autocorrelation to give a gain measuring correlation. For an individual pixel approaching 
zero defect where both the autocorrelation and crosscorrelation are zero, little is contributed 
by the pixel to the overall average for its region. On the other hand, at a pixel with a strong 

30 defect record, both the crosscorrelation and autocorrelation are large, and the corresponding 
pixel will contribute significantly to the average for its region. 

From the regional cross correlation and regional autocorrelation values, the 
correlation value is determined at step 194 in Fig. 9B. This process is further detailed in 
Figs. 15A and 15B. First, an articulation gain value which is the exact gain at a pixel 

35 required to null the visible record is calculated at step 196 by dividing the image element by 
the defect element. This is equivalent to dividing the single element crosscorrelation, or 
image element multiplied by the defect element, by the single element autocorrelation, or 
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defect element squared. As previously described, the correlation value is obtained by 
dividing the regional crosscorrelation by the regional autocorrelation at step 198. The 
threshold mask value is obtained by dividing the regional autocorrelation by a 
predetermined threshold constant at step 200. In this embodiment, the threshold constant is 
5 .015. If the autocorrelation, or defect record squared, exceeds this threshold for any pixel, 
then that pixel is allowed to "articulate" the required correction coefficient for complete 
cancellation of the visible record at that particular pixel, and change the visible image pixel 
by pixel, without regional averaging. Note that if the threshold mask is zero at step 202, 
the new upper cutoff will be equal to the new lower cutoff which is simply equal to the 

10 correlation as shown at step 204. If the threshold mask is one at step 206, the new upper 
cutoff equals the old upper cutoff, originally assumed to be 2.0, and the new lower cutoff 
equals the old lower cutoff, originally assumed to be 0.0 as seen at step 204. Correlation is 
set to articulation at step 208 if articulation is between new upper cutoff and new lower 
cutoff, but is limited to the range between new upper cutoff and new lower cutoff. In 

15 effect, if the defect intensity is low, correlation will be left unchanged, but if the defect 
intensity is high, then correlation will be equal to articulation. In other words, if the defect 
intensity is high, then the correlation will track articulation for complete nulling of the image 
in the visible record. 

The range of correction is still limited, however, to be close to the average to 

20 prevent extreme cancellations outside the range of probability. Using the threshold mask 
value, new upper and lower cutoff values are generated which serve to bound the 
correlation value generated above. This process is referred to in Fig. 9B as calculation of 
the bound correlation value at step 210 and is described in detail in Fig. 16. The upper 
cutoff and lower cutoff values may be made more restrictive than the full range 0.0 to 2.0. 

25 In particular, it has been found to improve reconstruction quality when only the largest 
pyramid levels have the full range, and the smaller levels are constrained to a narrow range 
based on values of correlation in corresponding areas of higher levels. Fig. 16 illustrates 
an algorithm to set the upper cutoff and lower cutoff to be plus and minus 20% from the 
actual correlation found for the previous larger pyramid level. 

30 As shown in Fig. 9B, this new bound correlation value is then used to correct the 

image element at step 212. The new correlation value is multiplied by the defect element 
and this product is subtracted from the image element. The image element is now fully 
corrected. After each image element is corrected in the manner described, reconstruction of 
the fullpass image begins as illustrated in Fig. 17. Starting from the bottom of the image 

35 pyramid, each level is upsized by adding each element of the upsized level to each element 
of the level below. After all the levels have been added, the reconstruction of the log image 
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is complete. Upon completion, the antilog is taken of the log image and the resulting image 
can now be displayed, stored, or transmitted. 

While this invention has been shown and described with reference to particular 
embodiments thereof, it will be understood by those skilled in the art that the foregoing and 
5 other changes in form and detail may be made therein without departing from the spirit and 
scope of the invention. 
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1 . A method for correcting defects in an image comprising: 

applying light to an image storing medium; 
5 receiving from the image storing medium an image signal indicating an image 

deteriorated by defects; 

applying a defect distinguishing radiation to the image storing medium; 

receiving from the image storing medium a defect signal, in register with the image 
signal, indicating the defects in the image; 
10 applying a gain to the defect signal to generate a normalized defect signal; 

differencing the image signal and normalized defect signal to generate a recovered 
image signal; and 

adjusting the gain to minimize the defects in the recovered image signal. 

15 2. The method of claim 1 wherein the image storing medium is a substrate bearing a 
two dimensional image. 

3 . The method of claim 2 wherein the substrate bears a visible image. 

20 4. The method of claim 2 wherein the defect signal indicates the locations and 
magnitudes of substrate defects. 

5. The method of claim 2 wherein the defect signal is received from the substrate in 
infrared light. 

25 

6. The method of claim 1 wherein the step of adjusting the gain to minimize the defects 
in the recovered image signal comprises the step of measuring the defects in the recovered 
image signal. 

30 7. The method of claim 6 wherein the step of measuring the defects in the recovered 
image signal comprises the step of calculating the correlation between the recovered image 
signal and the defect signal. 

8 . The method of claim 7 wherein the step of adjusting the gain further comprises the 
35 step of altering the gain as a function of the correlation between the recovered image signal 
and the defect signal. 
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9. The method of claim 1 wherein the defect signal includes a plurality of defect signal 
frequency bands, and wherein the gain is adjusted differently between at least two of the 
bands. 

5 10. The method of claim 9 wherein one of the defect signal frequency bands comprises 
the D = Factor of the image signal. 

1 1 . The method of claim 9 wherein the defect signal frequency bands are logarithmically 
spaced by a frequency incrementing factor. 

10 

1 2. The method of claim 1 1 wherein the frequency incrementing factor is two. 

13. The method of claim 9 wherein the defect signal frequency bands are linearly 
spaced. 

15 

14. The method of claim 9 wherein the step of adjusting the gain to minimize the defects 
in the recovered image signal comprises the step of calculating the correlation between the 
image signal and at least one of the defect signal frequency bands. 

20 15. The method of claim 14 wherein the step of adjusting the gain further comprises the 
step of calculating the gain as a function of the correlation between the image signal and at 
least one of the defect signal frequency bands. 

16. The method of claim 9 wherein the defect signal received is further characterized to 
25 include a plurality of defect signal regions, and wherein the gain is adjusted differently 

between at least two of the defect signal regions. 

17. The method of claim 16 wherein the step of adjusting the gain to minimize the 
defects in the recovered image signal comprises the step of calculating the correlation 

30 between the image signal and at least two of the defect signal regions. 

18. The method of claim 17 wherein the step of adjusting the gain comprises the step of 
calculating the gain as a function of the correlation between the image signal and at least two 
of the defect signal regions. 

35 

19. The method of claim 18 wherein the step of calculating the gain for a selected defect 
signal region within a selected defect signal frequency band comprises the steps of 
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calculating the autocorrelation of the defect signal within the selected defect signal region 
and the defect signal band, and the crosscorrelation of the defect signal within the selected 
defect signal region and the defect signal band with the image signal. 

5 20. The method of claim 19 wherein the gain within the selected defect signal region 
and the defect signal band is a function of said crosscorrelation divided by said 
autocorrelation. 

2 1 . The method of claim 16 wherein the defect signal received is further characterized as 
10 comprising at least one intermediate region overlapping at least two of the defect signal 

regions, and wherein the gain is further characterized as changing smoothly across at least 
two of the defect signal regions and the intermediate region. 

22. The method of claim 20 wherein the defect signal regions are further characterized 
15 as comprising of a plurality of points, and wherein the steps of calculating the 

autocorrelation and the crosscorrelation further includes the steps of calculating the point 
autocorrelations of the defect signal at the plurality of points and the point cross correlations 
between the defect signal and the registered image signal at the plurality of points, and 
averaging the point autocorrelations to obtain the autocorrelation within a region and 
20 averaging the point cross correlations to obtain the cross correlations within a region. 

23. The method of claim 22 wherein the averaging steps further include spatially 
lowpass filtering the point autocorrelations to produce a smudged autocorrelation signal 
containing smudged autocorrelation points, and spatially lowpass filtering the point cross 

25 correlations to produce a smudged crosscorrelation signal containing smudged 
crosscorrelation points. 

24. The method of claim 23 wherein the adjusted gain at a selected point of the defect 
signal is a function of the coiresponding smudged autocorrelation point divided by the 

30 corresponding smudged crosscorrelation point. 

25. An apparatus for correcting the effect of defects in an image comprising: 
means for applying light to the image storage medium; 

means for receiving from the image storing medium an image signal indicating an 
35 image deteriorated by defects in the image storage medium; 

means for applying a defect distinguishing radiation to the image storage medium; 
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means for receiving from the image storage medium a defect signal, in register with 
the image signal, indicating the defects in the image; 

means for applying a gain to the defect signal to generate a normalized defect signal; 
means for differencing the image signal and normalized defect signal to generate a 
5 recovered image signal; and 

means for adjusting the gain to minimize the defects in the recovered image signal. 

26. The apparatus of claim 25 wherein the recording medium is a substrate bearing a 
two dimensional image. 

10 

27. The apparatus of claim 26 wherein the substrate contains a visible image. 

28. The apparatus of claim 26 wherein the defect signal is received from the substrate in 
infrared. 

15 

29. The apparatus of claim 25 wherein the gain adjusting means includes means for 
measuring the defects in the recovered image signal. 

30. The apparatus of claim 29 wherein the measuring means comprises means for 
20 calculating the correlation between the recovered image signal and the defect signal. 

31. The apparatus of claim 25 wherein the gain adjusting means comprises means for 
altering the gain as a function of the correlation between the recovered image signal and the 
defect signal. 

25 

32. The apparatus of claim 25 wherein the gain adjusting means comprises means for 
measuring the defects in the image signal. 

33. The apparatus of claim 32 wherein the measuring means comprises means for 
30 calculating the correlation between the image signal and the defect signal. 

34. The apparatus of claim 33 wherein the gain adjusting means comprises means for 
calculating the gain as a function of the correlation between the image signal and the defect 
signal. 

35 
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35. The apparatus of claim 25 wherein the defect signal comprises a plurality of defect 
signal frequency bands, and wherein the gain is adjusted differently between at least two of 
the bands. 

5 36. The apparatus of claim 35 wherein one of the defect signal frequency bands is a 
mean. 

37. The apparatus of claim 35 wherein the defect signal frequency bands are 
logarithmically spaced by a frequency incrementing factor. 

10 

38. The apparatus of claim 37 wherein the frequency incrementing factor is two. 

39. The apparatus of claim 35 wherein the defect signal frequency bands are linearly 
spaced. 

15 

40. The apparatus of claim 35 wherein the gain adjusting means further comprises 
means for calculating the correlation between the image signal and at least one of the defect 
signal frequency bands. 

20 41. The apparatus of claim 40 wherein the gain adjusting means further comprises 
means for calculating the gain as a function of the correlation between the image signal and 
at least one of the defect signal frequency bands. 

42. The apparatus of claim 35 wherein the defect signal is further characterized by 
25 comprising a plurality of defect signal regions, and wherein the gain is adjusted differently 

between at least two of the regions. 

43. A method of processing visual image data with non-image defect data to remove 
non-image defect information from the visual image data, the non-image defect data being 

30 registered with the visual image data, comprising: 

creating, from the visual image data, a plurality of visual image data sets for 
respective spatial frequency bands; 

creating from the visual image data a plurality of non-image defect data sets for 
respective spatial frequency bands substantially similar to the spatial frequency bands for 
35 the visual imaging data; 
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cross correlating the visual image data set and the non-image defect data sets for 
respective spatial frequency bands to obtain crosscorrelation variables for each of the spatial 
frequency bands; 

autocorrelating the non-image defect data set for each of the spatial frequency bands 
5 to obtain autocorrelation variables for each of the spatial frequency bands; 

processing the crosscorrelation variables over a plurality of spatial regions to obtain 
spatially defined crosscoirelation variables for each of the spatial frequency bands; 

processing the autocorrelation variables over a plurality of spatial regions 
substantially similar to the spatial regions of the crosscorrelation variables to obtain spatially 
10 defined autocorrelation variables for each of the spatial frequency bands; 

processing the spatially defined crosscorrelation variables with the spatially defined 
autocorrelation variables to obtain spatially defined nulling gains for each of the spatial 
frequency bands; 

applying the spatially defined nulling gains to the respective non-image defect data 
15 sets to obtain nulling data sets for the respective spatial frequency bands; 

applying the nulling data sets to the visual image data sets for the respective spatial 
frequency bands to obtain corrected visual image data sets for the respective spatial 
frequency bands; and 

merging the corrected visual image data sets for the respective spatial frequency 
20 bands to obtain corrected visual image data. 

44. A method of processing visual image data with non-image defect data to remove 
non-image defect information from the visual image data, the non-image defect data being 
registered with the visual image data, comprising: 

25 comparing the visual image data with the non-image defect data to determine a 

nulling gain; 

applying the nulling gain to the non-image defect data to create nulling data; and 
applying the nulling data to the visual image data to remove non-image defect 
information therefrom. 

30 

45. The method of claim 43 further comprising identifying a plurality of spatial 
frequency bands to overcome limitations in real optics and image sensors, the visual image 
data further comprising a plurality of respective visual image data sets for the spatial 
frequency bands, the non-image defect data further comprising a plurality of respective 

35 non-image defect data sets for the spatial frequency bands, the nulling gain further 
comprising a plurality of respective nulling gain sets for the spatial frequency bands, and 
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the nulling data further comprising a plurality of respective nulling data sets for the spatial 
frequency bands. 

46. The method of claim 43 further comprising identifying a plurality of spatial regions 
of the image, wherein the nulling gain is determined for each spatial region. 
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Done 


FIG. 13 
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Calculate Clamped Element 


21/25 


Receive: 
image element 
defect element 
low clamp 
upper clamp 


f Clamping ^ 
\begins here./ 


Clamped element = image 
element 



ClampedE lenient = 
LowC larap*defect element 



^UpperClamp^ 
Nol 


No 


ClampedE lement= 
LowC lamp*defect element 


CtampedE lement = 
Uppe rC iamp^de fee t elenent 



FIG. 14 


■C 


ClampedE lement = 
UpperC lamp*defect element 


Done. 
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Calculate Correlation value 



Correlation calculation 
starts here. 


196^ 




Articulation = image element / defect element 

198 -\ 




Correlation = RegionalCrossCorrelation/ 
RegionalAutoCorre la t ion 


200^ 




ThresholdMask = RegionalAutoCorrelation/ 
THRESHOLD 




ThresholdMask = 0 


204 


NewUpperCutoff = (UpperCutoff*Thresho IdMask) + (Correlation * (1 - ThresholdMask)) 
NewLowerCutoff = (LowerCutof f»Thresho IdMask) + (Correlation ♦ (1 - ThresholdMask}) 


FIG. 15A 
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Bound Correlation Value 


24/25 


Bounding of 
Corre lat ion 
^begins here, 


AvgPrevCorr = Average of previous level Correlation values 


UpperAvgPrevCorr = AvgPrevCorr + (20% of AvgPrevCorr) 

LowerAvgPrevCorr = AvgPrevCorr + (20% of AvgPrevCorr) 
1 


Yes 



r- UpperAvgPrevCorr = UpperCutoff 


Yes 


LowerAvgPrevCorr = LowerCutoff 



Yes 



r- Correlation = UpperAvgPrevCorr 


Yes 


Correlation = LowerAvgPrevCorr 
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Reconstruct Fullpass Image 


c 


Reconstruction 
begins here 


3 


= NumOfLevels-1^ 


For each level of the image pyramid starting from 
the 'bottom* (i.e. the level with dimensions 1x1). 


Upsize level[i] 


Add each element of the upsized 
leve l[ i] to each element of level[i-1] 
and save the result in level[i-1] 


i = i-1 



i>0 
No 


Yes 


Done. The final 
reconstructed image 

is level[0] 
image pyramid 


FIG. 17 
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